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The growing influence of academic rankings 


Over the last decade, rankings have become a significant force impacting on and influencing higher education, students and 
parents, policymakers, and other stakeholders. While they have been part of U.S. academic system for 100 years, global 
rankings are a more recent phenomenon. 


The arrival of global rankings coincided with the intensification of competition in the international economy and increasing cross- 
border student mobility. They have also fulfilled an information deficit. While there are various evaluation and benchmarking 
instruments around, there has been growing dissatisfaction with the robustness of these tools. By placing higher education 
institutions (HEIs) within a wider international and comparative framework, rankings have managed to say something about 
quality in a simple, accessible and provocative way. 


Today, there are ten major global rankings. The most well-known are: Academic Ranking of World Universities (Shanghai Jiao 
Tong University, China), QS World University Rankings (Quacquarelli Symonds, UK), THE World University Ranking (Times 
Higher Education, UK), and U-Multirank (European Commission, Brussels). U.S. News and World Report Best Colleges has 
been around since the mid-1980s, and recently launched its global version, Best Global Universities Rankings. 


Around the world, there are over 150 country-based or specialist rankings — and the number is growing. This includes rankings 
which measure and compare performance at the institutional and discipline level, and those which focus on societal impact, 
value-for-money and affordability, environmental awareness, etc. Some rankings specialize on regional comparisons for Asia, 
Latin America or Middle East and North Africa countries. There are also a small number of system-level rankings. With some 
exceptions, most rankings are commercial operations. 


What are Rankings? 


Rankings work by comparing HEls using a range of indicators, which are usually weighted according to priority. There is no 
such thing as an objective ranking; the choice of indicators and their weightings reflect the value judgements of the ranking 
organisation. Each indicator is considered independently from each other with no consideration as to context, history, mission, 
etc. The final score is aggregated to a single digit in descending order, often referred to as a “league table." 


Rankings are used to inform student choice and stakeholder opinion, and to assess the performance of scientific-scholarly 
research. Research has shown that students, public opinion and government are the biggest users of rankings. For a 
government, doing well in rankings can heighten a country’s status, and help attract foreign-direct investment and talented 
students and professionals. Because of these consequences, rankings have become inculcated into higher education strategic 
decision-making and institutional research. 


While rankings are widely used they are also broadly critiqued. One of the main criticisms is that rankings use the same set of 
indicators to measure HEls operating in very different national settings and meeting a diverse set of needs. This undermines 
mission distinctiveness, ignores diversity amongst the student cohort, and promotes a single model of higher education 
excellence. Rankings are a hierarchical system, signalling some HEls or disciplines are more important than others. But, 
academic quality is not easily reduced to measures of quantification. 


Do Rankings Measure Quality? 


Rankings purport to measure and compare higher education performance and “quality.” But, determining “which university is 
best” depends upon who is asking and why. After-all, there are over 18,000 HEIs worldwide, but rankings focus predominantly 
on the top 100 or less than 0.5 percent. 


Global rankings usually focus on bio- and medical science research; student and faculty characteristics (e.g. research 
productivity, entry criteria, faculty/student ratio); internationalization; and reputation amongst peers, employers and students. 
They do not include measures of teaching and learning; arts, humanities and social science research; regional or civic 
engagement; and the student experience. National rankings are broader in scope because of the existence of and access to an 
array of data sources. 


To get around problems of inadequate data, rankings use proxies. But, this can easily lead to simplistic comparisons and 
conclusions. A few examples: 


¢ Student entry scores are often used to gauge the quality of the student, but research has shown that entry scores simply 
reflect socioeconomic advantage and say nothing about quality of the higher educational experience itself. Student 
completion or graduation rates are similarly influenced by student profile characteristics. 


The faculty/student ratio is regularly used as a proxy for teaching quality. But, evidence on the impact of class size on 
student performance is mixed, and the data is open to “gaming.” There are differences across disciplines and types of 
learning environments, and for public and private institutions and systems. Ultimately, the faculty/student ratio may say 
more about the funding or efficiency level than the quality of teaching. 


The size of the budget or the library is often considered a proxy for the quality of the learning environment. But this 
provides little information about how these resources are used or how students benefit. At a time of massive expansion in 
higher education provision around the world and increasing use of digital resources, measuring expenditure can penalize 
developing countries and newer HEls. 


Reputational surveys are increasingly used to garner world-wide opinion, but responses are very subjective and the 
response rate is very low. They benefit older institutions in developed countries and global cities with which there is easy 
identification. It is not possible to assess institutional or teaching quality in this way. 


Rankings and Quality Assurance 


Over the last decade, there has been an explosion in what are euphemistically called transparency and accountability 
instruments and tools, operating in tandem but differing with respect to purpose, policy orientation and user. Rankings are part 
of this trend but they differ substantially from quality assurance (QA). 


Quality assurance derives from a strong ethos of institutional autonomy. QA relies upon peer review conducted at the 
institutional or sub-institutional level. It embraces a strong commitment to quality improvement and enhancement rather than 
measurement. 


Today, traditional “self-regulating” aspects of academic QA are coming under scrutiny. From an external perspective, the 
language used in the reports can be difficult to interpret and institutional performance is not easily comparable, especially 
internationally. There is also a growing public interest in going beyond simply measuring and evaluating quality to linking 
performance and productivity to resource allocation. Accordingly, there has been a tendency to integrate rankings with other 
QA processes or to use them for benchmarking, accreditation and/or exposing poor performance. 


What’s Next? 


Quality and excellence are key differentiators in the national and global market. Whether we like rankings or not, they have 
succeeded in drawing attention to how we assess and compare HEls and what these assessments mean in terms of quality. 
In a globalized world, students, graduates and employers, and society overall, require confidence. 


The depth of the global economic crisis has focused attention on the capacity of HEIs to meet the needs of national economies, 
and to respond to concerns about graduate employability and affordability. Drawing on the Bologna experience, there has been 
a noticeable shift to measuring teaching and learning outcomes that allow us to judge whether graduates have the threshold 
qualities we expect. Institutional and country-based evaluations are rapidly being overtaken by international efforts. Rankings 
initially filled this gap but they focus too narrowly on elite universities and research. 


Alternatives such as U-Multirank and U21 Ranking of National Higher Education Systems embrace a greater diversity of HEls, 
and look at the over all capacity of higher education to bring benefit to society, respectively. On-line country-level profiling tools 
are being developed to provide quick comparative data on university performance and productivity. In the United States, the 
Postsecondary Institution Rating System (PIRS) aims to link access, affordability and outcomes. With the exception of the 
European Tertiary Education Register (ETER), several commercial initiatives are busy monetising higher education data. Social 
media sites may morph into a TripAdvisor-like service. 


In the early days, the emphasis was on quality assurance and student choice; today, the ground is shifting between autonomy 
and accountability, and between steering and regulation. In response, two actions are essential. i) Higher education should 
become more actively engaged in the global conversation about quality, and identify meaningful measures which can 
demonstrate value and contribution. ii) Higher Education, along with key stakeholders, should agree upon a common 
international database to be held by a not-for-profit international organisation. 
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